Hi there i was wondering if there is a more efficient way of removing empty data than this.
Hi @AbdulBasit
Yes, you can remove rows that have nulls in any column using the following method,
Here is a modification which will improve the performance of @PhilipMannering workflow
Since if one column has null row it should be removed. So if one ID is there in Null() group it should be removed.
Basically transpose might create duplicates of IDs like below.
Which when it goes to Join it might create multi join results to data explosion.
So what i did is i have added a unique tool which will take single ID only once in the Null group to join and reduce data explosion. Since you said 50 columns you could use all the help you can get😀
Hope this helps 🙂
While you could put a custom filter that combines all of the empties that you want to find, this way is actually more efficient. Every time you filter, you're making a smaller dataset. So the next filter will run faster and so on.
Let me know if that helps
Cheers!
Esther
@PhilipMannering , nice!! Did you check speed on both approaches?
for a data set having to many columns this is more efficient other wise the canvas would get messy and if we have 50 colums lets say it would not be a good idea to add 50 filters.
for a data set having 50 colums i think this would be efficient. Tell me if we have 50 columns and we want to remove nulls from them will this method work without adding any other tool.
Can you show me the way.
