Hi all,
I think there must be a better way to achieve what we are doing, but struggling to implement it right now:

This is the start of our process - what it is basically doing is isolating any new records in two SQL tables before they are sent through other processes. But i think it is extremely in-effecient. It imports the entirety of both tables and then de-dupes against already processed records stored in an xydb file.
What i need i think is to use a dynamic in-db filter to only extract the data needed.
Just been getting my head round batch macros and have built this:


Which i think is doing what i need (working out what data it needs and then updating the filter to select that data). However this is not in-db which means i still need to extract the whole table, which is what i am trying to avoid.
I'm a bit baffled as to what the in-db macro in/out is and does - i think if i could do the above, but for in-db, that would probably solve my issue.
Any suggestions would be very gratefully received!