I am building an iterative macro that can dynamically clean data from a raw data set for predictive analytics, but as part of it, I would like to de-select fields that contain exactly 1 distinct non-null value (or said another way, only select fields with at least 2 distinct non-null values). To do this logic, I'm using the Frequency Tool in combination with concatenation aggregations, filters, and dynamic selects to figure out which fields to keep and which to remove.
The macro internally works like it is supposed to using the macro input file I used to build it. However, my problem comes from trying to use the macro and push a new data set into the macro (externally from the macro); any other data set I try to use yields null outputs. After putting output files everywhere, I've narrowed down the problem to be defined as the following: the Frequency Tool does not seem to be dynamic enough to be used in a macro. It can't handle new fields and yields only null values because it looks only for the macro input field names and doesn't allow the new data set field names to pass through.
Am I doing something wrong, or do I need to try to dynamically update the XML for the Frequency Tool somehow? This feels like it should have already been solved, but maybe the frequency tool usage is a new case.
Thanks in advance for any help!
