I have a folder containing a variety of files. According to business requirements, I need to run a particular workflow (quite easy) on all csv files in the folder. I am assured that they will be of the same type. Further, my output filename shall be a function of the original file name and they need to be slotted into a folder-hierarchy, based on date of running the workflow.
So my overall workflow (image attached) goes as:
(1) Run directory tool for *.csv.
(2) Sample the top 1 row.
(3) Feed to dynamic input
(4) And, run the individual data-processing workflow.
(5) From the result of (2), use formula tool to get new file name, folder name, and path. Run python tool to create the folder.
(6) Append result of (5) with result of (4), remove extraneous metadata columns except for path, and write to output (by replacing file path) using path column.
(7) Provided a successful output (block tool), move the processed file to a separate directory using python tool.
I am wondering about how to loop my overall workflow infinitely until Step (1) returns no row. Probably, iterative macro is the way to go about this but I am a total noob in it. Will appreciate some help.
