Hello community, I am working on a workflow as following structure:
Data input - Filter to Workcenter: "WorkcenterA" - Transformation (a big chunk of transformation/calculation) - output
- Filter to Workcenter: "WorkcenterB" - Transformation (a big chunk of transformation/calculation) - output
- Filter to Workcenter: "WorkcenterC" - Transformation (a big chunk of transformation/calculation) - output
- Filter to Workcenter: "WorkcenterD" - Transformation (a big chunk of transformation/calculation) - output
...... and so on, at the end I need to union all of their output, that is my goal
(I cannot skip the Filter to workcenter part, because it will cause the calculation result differently)
I have about 40 unique workcenter that i need to do this way, after I copy and pasted those big chunk of 5-6 of them I found that workflow is getting very slow, and I heard that the Batch Marco can achieve the goal with only one big chunk of transformation and loop through workcenter and union at the end, how to do that?
I hope these make sense, Its hard to provide a sample dataset, but I will try:
The sample input:
Workcenter Order Metric_A Metric_B Metric_C
| Line1 | U001 | 2 | 5 | 6 |
| Line1 | U002 | 3 | 4 | 7 |
| Line2 | U003 | 5 | 6 | 9 |
.....
Output:
Workcenter Metric Stat Value
| Line1 | Metric_A | Min | 2 |
| Line1 | Metric_A | Avg | 3.0 |
| Line1 | Metric_A | Max | 5 |
| Line1 | Metric_B | Min | 4 |
| Line1 | Metric_B | Avg | 4.5 |
....
No need to work on the transform part, I have already done, the key point is to avoid filter each single one of the Workcenter and use Batch Marco to 'loop' through to reduce the workflow complexity and improve its performance.
Thanks for your help
