Iterative Macros on Large Data Sets like Azure via MS Graph APIs

[jrobiso2]

Given that the current Iterative type Macro will hold each iteration output in memory, to merge it all at the end of the iterations, wouldn't it make sense to have a checkbox for "Large Data Sets" where each iteration's output is stored to a tmp file, to be merged after the iterations stop?

As it is, holding multiple iterations in memory may often be too much data for a PC, no matter how hefty.  By off-loading the memory objects to a tmp file and only holding the currently working iteration in memory, speed and efficiency would be gained.

This is especially critical when using Iterative Macros to cycle through APIs (pagination) and while each individual data pull may be small enough, they quickly grow in memory as each data pull is added to the in-memory object.