Column Filter

[Dcurwenreed]

I suggest that it would be beneficial to add in a column filter that can automatically remove columns based on a condition, such as removing columns where all values are NULL or if they contain something in the values.

Should have a True and False output, like the normal Filter tool, so you can check what is being removed.

e.g. Would help with when you get poorly formatted excel sheets that add in hundreds of redundant columns, or if your workflow has generated NULL columns that should be removed, without having to Transpose, Filter, Cross tab etc to clear them out.

Thanks,

Doug

[PangHC]

it may work by this way, with multiple tool. (idea of tools)  

1. More detail in field summary tool

Field summary tool should have data in the browse tool data (like count of non-nulls, and count of nulls)

  hence, we can use normal filter tool to have the condition we want.

2. New tool that able filter column by row value.

It should have 2 input anchor, 1 or 2 output anchor

Input 1: Raw Data,

Input 2: Data from field summary tool above (after filter).

1 output; or

output: Data that keep/remove (based on configuration set) column that in Input 2.

2 output

output 1: Input 1 with column that list in Input 2. (i.e Only Field1 is keep)

output 2: Input 1 without column that list in Input 2. (i.e. Other than Field1)

Configuration: 

1. Keep or remove

2. Which field in input 2 is header.

[Joe_Lipski]

The new updates to the Data Cleanse tool remove columns where every value is NULL