Master Workflow pipeline with managing dependencies

[narula8]

Hello Experts,

Reaching out to you for some pointers on below topics.

1.In our current project , we are following medallion architecture (i.e., we have data in three layers - BRONZE, SILVER, AND GOLD tables).
We are in need to create a master pipeline which can load data end to end managing dependencies.

Logic for defining dependencies  : Based on the total records fetched in current load.

2.If no of records in latest run are more than previous run continue, else wait for 30 min and rerun the flow.

BTW are loading data using APIs.

Any inputs will be appreciated.