Pre-Filter in Input Tool

[Pawel_Paleczny]

Pre-Filter as new option in Input Tool might reduce import data and allows to input only selected data (ie. for specific period or meeting certain conditions).

Cheers,

Pawel

[cgoodman3]

I think this would be a really handy feature when you have a very large dataset. But I have a query around whether it is possible when you read the data into memory. How do you know you have a complete read of the data from the file. Would you prefer to have the read of all the data and then a filter tool immediately after, so you can check if on the false output there are any misspelt records for example Sun instead of Sunday?

Also are you looking at doing this for speed of development? If so would sampling your data get the improvement you want, so instead of having the option of only reading the first x records, that you could opt to read in a random 10% of records?

[Pawel_Paleczny]

thanks Chris for your reply! as you mentioned that would be helpful for large datasets - instead of loading milions of rows you can download specific set of the data (similar to SQL clause WHERE). Filter tool after Input tool is good solution however you still need to read all records - even those which are not necessary in further analysis. Re misspelt records it should be case sensitive in my opinion.

Reading first x rows just reads rows without any other conditions so I think that's not scope of this idea.

[marcusblackhill]

That can be useful for large dataset as mentioned. Going a bit further, that "pre-filter" idea with a "select tool" embedded would be interesting, because some cases you have far more columns than you really need from a dataset or in other situations, where Alteryx import certain fields in a different data type than you expect, so you could just set the data types to be used in the import with a feature like that.

[Pawel_Paleczny]

@marcusblackhill embedded Select Tool in Input sounds incredibly great!