Engine Improvement: Prune unused fields

[SeanAdams]

We see canvasses every day where dozens fields are brought into a canvas or a macro, but never used - and this just creates slowness for no good benefit.

Given that one of the selling features of Alteryx is the speed of processing  - could we look at three improvements to the Alteryx engine & designer:

• easiest: Keep track of every field brought in / created - and if they are not used in an output, then throw a warning at the end of the execution process
  • For example - you bring in fields a,b,c - you create field d and e during the flow in formula tools
  • Field d is never used as an input to any filters or formulae - and it doesn't appear on any output - so it's just waste
  • Field a and b are part of the output, so they are fine
  • Field c is never used at all - so that's just waste.
  • Field e is used to filter the records before output - so this one is fine.
  • So we've immediately found 2 fields that we can eliminate and make this canvas faster
• Medium: Ignore the unused fields in the execution engine
• Hardest: Tell the users that their field is unused in Alteryx Designer by doing a lineage analysis of the tools, just like software environments like Visual Studio do.    This may require a change to the engine & to designer 'cause we would need to make each tool capture the full detail of the fields that they know in their configuration in order to do this trace.

[MarqueeCrew]

@SeanAdams ,

 I'll add extra credit:

 often data is passed to outputs but is not referenced.  Speed is improved if this dead weight is not part of process. 

cheers

[erich_shiino]

@SeanAdams and @MarqueeCrew  , 

Developers would get a report at the end with all the fields that were actively used (filters, formulas, joins, etc), 

Another report would highlight fields that are passively used (they are part of the outputs and nowhere else).

And of course, fields that are not used anywhere. 

We should have the option to remove fields on any of those three categories with a mouse click. 

Maybe it could be part of the Performance profiling analysis.

[Thableaus]

+1 on this idea. This could be part of the AI enhancements to make the tool smarter. Something like "Ask Data" from Tableau, but done better.

[CristonS]

thanks for posting @SeanAdams and everyone! I remember a million years ago when I was in customer support, one of the first "best practices" tips we gave to new users was to deselect all the fields that you didn't need, in order to speed up processing. It makes more sense for the platform to do this for users. Good stuff!

[Hollingsworth]

Yes to this! Thank you Sean for a great idea.

[MarqueeCrew]

@Hollingsworth Nice point to make. 

[TonyaS]

Thanks @SeanAdams 

I wanted you to know that this has been added to our internal Intake for product ideas.